Docket No. SAI.P001 US 

KBREF. NO.: 1023 POO 1 US 



WHAT IS CLAIMED IS : 

1 . A display system for a device, comprising: 
computer executable instructions operable to: 

receive images from a first sensor and a second sensor representing 

scenery outside the device; 
detect moving objects in the images; 
fuse the images to a single viewpoint; and 

transform the fused image to a first viewpoint image from a first operator 
station in the device and a second viewpoint image from a second 
operator station in the device, wherein the first and second 
viewpoint images conform to the scenery outside the device from 
each operator station. 

2. The display system of Claim 1 , further comprising: 
computer executable instructions operable to: 

combine the first and second viewpoint images with symbols, wherein the 
symbols represent information regarding the operational state of 
the device and the moving objects detected in the images. 

3. The display system of Claim 1, wherein the instructions for detecting 
moving objects in the first sensor image are configured to execute in a first processor, and 
the instructions for detecting moving objects in the second sensor image are configured to 
execute in a second processor simultaneously with the instructions in the first processor. 

4. The display system of Claim 3, wherein the instructions for transforming 
the fused image to the first viewpoint image are configured to execute in the first 
processor, and the instructions for transforming the fused image to the second viewpoint 
image are configured to execute in the second processor. 

5. The display system of Claim 2, wherein the symbols represent the moving 
objects in the vicinity of the device. 
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6. The display system of Claim 2, wherein at least one of the first and second 
viewpoint images include environmental information for the area where the device is 
operating. 

7. The display system of Claim 2, wherein the symbols represent weather 
hazards in the vicinity of the device. 

8. The display system of Claim 2, wherein the computer executable 
instructions are further operable to receive an enhanced image from a third sensor 
configured to provide an image of the out-the-window scenery in low-visibility 
conditions 

9. The display system of Claim 8, wherein the computer executable 
instructions are further operable to fuse the single viewpoint image with the enhanced 
image. 

10. The display system of Claim 9, wherein the computer executable 
instructions are further operable to utilize data from at least one position sensor to 
determine the location of the objects with respect to the device. 

1 1 . The display system of Claim 2, wherein the computer executable 
instructions are further operable to utilize data from off-board data sources regarding the 
objects. 

12. The display system of Claim 1, wherein the first sensor and the second 
sensor are video cameras. 

13. The display system of Claim 8, wherein the third sensor is a RADAR 

sensor, 

14. The display system of Claim 8, wherein the third sensor is a FLIR sensor. 
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15. The display system of Claim 2, wherein the computer executable 
instructions are further operable to generate a common display area associated with at 
least two mutually exclusive windows of information on the display device, wherein the 
common display area can be customized by the operator to display detailed information 
related to the information displayed in the associated windows. 

16. A method for providing an out-the-window visual scene on a display 
device, comprising: 

receiving an image of a portion of the out-the-window visual scene from the 

viewpoint of a first type of sensor; 
receiving another image of a portion of the out-the-window visual scene from the 

viewpoint of another of the first type of sensor; 
fusing the images from the first type of sensors into a combined image to generate 

a first fused image; 
transforming the fused image to a first operator viewpoint and to a second 

operator viewpoint; and 
outputting the first operator viewpoint image to a first display device and the 

second operator viewpoint image to a second display device, wherein the 

display devices are positioned to provide the portion of a desired out-the 

window visual scene in combination with a window that provides another 

portion of the desired out-the-window visual scene. 

17. The method of Claim 16, further comprising detecting objects in the first 
fused image from the first type of sensor. 

18. The method of Claim 1 7, further comprising combining the first fused 
image with symbols representing the objects. 

19. The method of Claim 16, further comprising transforming the first 
operator viewpoint image and the second operator viewpoint image to conform to the out- 
the-window visual scene. 
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20. The method of Claim 16, further comprising fusing the first fused image 
with an enhanced image of a portion of the out-the-window scenery from at least one of 
the group of a RADAR sensor and a FLIR sensor, to generate a second fused image. 

21. The method of Claim 16, further comprising fusing the second fused 
image with an enhanced image of a portion of the out-the-window scenery from at least 
one of the group of a RADAR sensor and a FLIR sensor, to generate a second fused 
image. 

22. The method of Claim 21, further comprising: transforming the second 
fused image to the first operator viewpoint and to the second operator viewpoint. 

23. The method of Claim 20, further comprising: providing portions of the 
transformed image with data from a terrain map database. 

24. A device, comprising: 
a display device; and 

a display processor operable to: 

receive a first sensor image representing a portion of scenery outside the 
device; 

transform the first sensor image to a viewpoint image from an operator 
station in the device, wherein the viewpoint image is sized and 
oriented to conform to the scenery outside the device from the 
operator station; and 

output the first operator viewpoint image to the display device, wherein the 
display device is positioned to provide the portion of a desired out- 
the window visual scene in combination with a window that 
provides another portion of the desired out-the-window visual 
scene. 
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25. The device of Claim 24, wherein the display processor is further operable 
to combine the viewpoint image with symbols, wherein the symbols represent 
information regarding the operational state of the device and the moving objects detected 
in the images. 

26. The device of Claim 24, wherein the display processor is further operable 
to detect moving objects in the first sensor image 

27. The device of Claim 24, wherein the display processor is further operable 
to generate symbols representing moving objects in the sensor image and the operational 
state of the device. 

28. The device of Claim 24, wherein the display processor is further operable 
to generate symbols representing weather hazards in the vicinity of the device. 

29. The device of Claim 24, wherein the display processor is further operable 
to receive an enhanced image of the out-the-window scenery in low-visibility conditions 
from a second sensor. 

30. The device of Claim 29, wherein the display processor is further operable 
to fuse the viewpoint image with the enhanced image. 

3 1 . The device of Claim 26, wherein the display processor is further operable 
to utilize data from at least one position sensor to determine the location of the objects 
with respect to the device. 

32. The device of Claim 26, wherein the display processor is further operable 
to utilize data from off-board data sources regarding the objects. 

33. The device of Claim 24, wherein the sensor is a video camera. 

34. The device of Claim 29, wherein the second sensor is a RADAR sensor. 

35. The device of Claim 29, wherein the second sensor is a FLIR sensor. 
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36. The device of Claim 24, wherein the display processor is further operable 
to generate a common display area associated with at least two mutually exclusive 
windows of information on the display device, wherein the common display area can be 
customized by the operator to display detailed information related to the information 
displayed in the associated windows. 

37. An aircraft, comprising: 

a crewstation with cockpit windows; 

a first display device for one crewmember; 

a second display device for another crewmember; and 

a display processor operable to: 

receive an image of an out-the-window visual scene from the viewpoint of 

a first type of sensor; 
receive another image of a portion of the out-the-window visual scene 

from the viewpoint of another of the first type of sensor; 
fuse the images from the first type of sensors into a combined image to 

generate a first fused image; 
transform the fused image to a first operator viewpoint and to a second 

operator viewpoint; 
transform the first operator viewpoint image and the second operator 

viewpoint image to conform to the size and orientation of the out- 
the-window visual scene; and 
output the first operator viewpoint image to the first display device and the 
second operator viewpoint image to the second display device. 

38. The aircraft of Claim 37, wherein the display processor is further operable 
to detect objects in the first fused image from the first type of sensor. 

39. The aircraft of Claim 38, wherein the display processor is further operable 
to combine the first fused image with symbols representing the objects and primary flight 
information for the aircraft. 
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40. The aircraft of Claim 37, wherein the display devices are positioned to 
provide the portion of a desired out-the window visual scene in combination with a 
cockpit window that provides another portion of the desired out-the-window visual scene. 

4 1 . The aircraft of Claim 37, wherein the display processor is further operable 
to fuse the first fused image with an enhanced image of a portion of the out-the-window 
scenery from at least one of the group of a RADAR sensor and a FLIR sensor, to generate 
a second fused image. 

42. The aircraft of Claim 4 1 , wherein the display processor is further operable 
to fuse the second fused image with an enhanced image of a portion of the out-the- 
window scenery from at least one of the group of a RADAR sensor and a FLIR sensor, to 
generate a second fused image. 

43. The aircraft of Claim 42, wherein the display processor is further operable 
to transform the second fused image to the first operator viewpoint and to the second 
operator viewpoint. 

44. The aircraft of Claim 43, wherein the display processor is further operable 
to provide portions of the transformed images using data from a terrain map database. 

45. The aircraft according to Claim 37 further comprising: 

a terrain database coupled to provide the display processor with at least a portion of the 
required out-the-window field of view on the display device. 

46. The aircraft of Claim 37, wherein the display processor is further operable 
to display one of the operator viewpoint displays to the operator acting as pilot in 
command of the aircraft during a predefined aircraft operational state, and to allow the 
pilot in command to choose an option on the display device to view detailed information 
about the aircraft and aircraft subsystems during other aircraft operational states. 
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47. The aircraft of Claim 46, wherein the display processor is further operable 
to generate a common display area associated with at least two mutually exclusive 
windows of information on each display device, wherein the common display area can be 
customized by the operator to display detailed information related to the information 
displayed in the associated windows. 
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